Skip to content

Optimize encode32 and decode32#10

Open
colinbendell wants to merge 2 commits intoabachman:masterfrom
Shopify:cb_encode_cleanup
Open

Optimize encode32 and decode32#10
colinbendell wants to merge 2 commits intoabachman:masterfrom
Shopify:cb_encode_cleanup

Conversation

@colinbendell
Copy link

@colinbendell colinbendell commented May 24, 2024

This PR focuses on optimizing the memory allocation for the byte manipulation to and from the Rockford Base32 encoding and bytes. Specifically the encode32 and decode32 can better utilize the standard rfc4648 base32 encoding for better performance.

Encoding performance:

Calculating -------------------------------------
   encode32_baseline    321.385k (± 1.8%) i/s -      1.612M in   5.016794s
  encode32_optimized      1.100M (± 6.3%) i/s -      5.542M in   5.074883s
     encode32rfc4648      1.894M (± 2.4%) i/s -      9.512M in   5.025403s

Comparison:
     encode32rfc4648:  1893996.2 i/s
  encode32_optimized:  1099515.7 i/s - 1.72x  (± 0.00) slower
   encode32_baseline:   321385.0 i/s - 5.89x  (± 0.00) slower

Calculating -------------------------------------
   encode32_baseline     1.336k memsize (     0.000  retained)
                        29.000  objects (     0.000  retained)
                        18.000  strings (     0.000  retained)
  encode32_optimized   479.000  memsize (    80.000  retained)
                        10.000  objects (     2.000  retained)
                         4.000  strings (     0.000  retained)
     encode32rfc4648   373.000  memsize (    80.000  retained)
                         8.000  objects (     2.000  retained)
                         3.000  strings (     0.000  retained)

Comparison:
     encode32rfc4648:        373 allocated
  encode32_optimized:        479 allocated - 1.28x more
   encode32_baseline:       1336 allocated - 3.58x more

Decoding performance:

Calculating -------------------------------------
   decode32_baseline    243.968k (± 2.4%) i/s -      1.240M in   5.084222s
  decode32_optimized      1.310M (± 9.4%) i/s -      6.722M in   5.218875s

Comparison:
  decode32_optimized:  1310294.2 i/s
   decode32_baseline:   243967.9 i/s - 5.37x  (± 0.00) slower

Calculating -------------------------------------
   decode32_baseline     1.800k memsize (     0.000  retained)
                        41.000  objects (     0.000  retained)
                        18.000  strings (     0.000  retained)
  decode32_optimized   267.000  memsize (     0.000  retained)
                         6.000  objects (     0.000  retained)
                         2.000  strings (     0.000  retained)

Comparison:
  decode32_optimized:        267 allocated
   decode32_baseline:       1800 allocated - 6.74x more

Also even simple time parsing can be optimized:

Calculating -------------------------------------
 time_parse_baseline      1.001M (± 7.3%) i/s -      5.007M in   5.051415s
time_parse_optimized      2.122M (± 7.4%) i/s -     10.511M in   5.028573s

Comparison:
time_parse_optimized:  2121531.7 i/s
 time_parse_baseline:  1000624.8 i/s - 2.12x  (± 0.00) slower

Calculating -------------------------------------
 time_parse_baseline   462.000  memsize (     0.000  retained)
                         8.000  objects (     0.000  retained)
                         2.000  strings (     0.000  retained)
      time_parse_new   286.000  memsize (    40.000  retained)
                         6.000  objects (     1.000  retained)
                         2.000  strings (     0.000  retained)

Comparison:
      time_parse_new:        286 allocated
 time_parse_baseline:        462 allocated - 1.62x more

@colinbendell colinbendell force-pushed the cb_encode_cleanup branch 2 times, most recently from 3a93119 to f04bdf7 Compare May 24, 2024 15:15
@colinbendell colinbendell mentioned this pull request May 24, 2024
Closed
@colinbendell colinbendell mentioned this pull request Jun 5, 2024
Closed
casperisfine
casperisfine reviewed Jun 5, 2024
View reviewed changes
@martelogan martelogan mentioned this pull request Jun 10, 2024
Merged
@colinbendell @casperisfine
Update lib/ulid/generate.rb
bac99a2 
Co-authored-by: Jean byroot Boussier <jean.boussier+github@shopify.com>
@martelogan martelogan mentioned this pull request Jun 10, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@paracycle paracycle paracycle left review comments

@casperisfine casperisfine casperisfine left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@colinbendell @paracycle @casperisfine